S.N. 10/033,597 
Art Unit: 2621 

IN THE CLAIMS 

1 . (Original) A computer implemented method for detemiining the semantic meaning of 
images, comprising: 

deriving a set of perceptual semantic categories for representing important 
semantic cues in the human perception of images, where each semantic category is modeled 
through a combination of perceptual features that define the semantics of that category and 
that discriminate that category from other categories; and 

for each semantic category, forming a set of the perceptual features as a 
complete feature set CFS. 

2. (Original) A method as in claim 1, wherein the perceptual features and their combinations 
are derived through subjective experiments performed with human observers. 

3. (Original) A method as in claim 1, further comprising extracting perceptual features from 
an input image and applying a perceptually-based metric to determine the semantic category 
for that image. 

4. (Original) A method as in claim 3, comprising processing the input image to compute the 
CFS; comparing the input image to each semantic category through the perceptually-based 
metric that computes a similarity measure between the features used to describe the semantic 
category and the corresponding features extracted from the input image; and assigning the 
input image to the semantic category that corresponds to a highest value of the similarity 
measure. 
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5. (Original) A method as in claim 1, further comprising computing features from the CFS for 
images in an image database; and generating a distance measure for characterizing a 
relationship of a selected image to another image in the image database by applying a 
perceptually-based similarity metric. 

6. (Original) A method as in claim 5, where values of the similarity metric computed for 
images in the image database are subsequently used to search for similar images in the image 
database. 

7. (Original) A method as in claim 5, where values of the similarity metric computed for 
images in the image database are subsequently used to organize images in the image database. 

8. (Original) A method as in claim 5, where values of the similarity metric computed for 
images in the image database are subsequently used to display images in the image database 
in an organized manner. 

9. (Original) A method as in claim 5, and further comprising defining a subset of features for 
the selected image or for an image retrieved fi-om the image database, and using the subset of 
features to refine a search through the image database. 

10. (Original) A method as in claim 5, wherein the image database is located at a remote 
location and is reachable through a data communications network. 

1 1 . (Original) A method as in claim 5, wherein the image database is located at a remote 
location and is reachable through a data communications network, and where the step of 
characterizing the relationship of the selected image to another image in the image database 
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by applying the perceptually-based similarity metric is accomplished to retrieve an image 
from the remote image database. 

12. (Original) A method as in claim 5, wherein the image database is located at a remote 
location and is reachable through a data communications network, and where the step of 
characterizing the relationship of the selected image to another image in the image database 
by applying the perceptually-based similarity metric is accomplished in conjunction with a 
text-based search algorithm to retrieve a multi-media object from the remote location. 

13. (Original) A method as in claim 3, wherein to assign a particular semantic category to an 
image all of a set of Required Features must be present in the image, and at least one of a set 
of Frequently Occurring Features must be present in the image. 

14. (Original) A data processing system comprising a data processor, a graphical user 
interface and a memory that stores a collection of digital images in an image database, said 
data processor operating in accordance with a stored program for determining the semantic 
meaning of images in accordance with a set of perceptual semantic categories that were 
previously derived from human observers and that represent important semantic cues in the 
human perception of images, where each semantic category is modeled through a 
combination of perceptual features that define the semantics of that category and that 
discriminate that category from other categories, where for each semantic category there 
exists a set of the perceptual features as a complete feature set CFS, said data processor 
extracting perceptual features from an input image and applying a perceptually-based metric 
to determine the semantic category for the input image. 

15. (Original) A system as in claim 14, where said data processor processes the input image 
to compute the CFS; compares the input image to each semantic category through the 
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perceptually-based metric that computes a similarity measure between the features used to 
describe the semantic category and the corresponding features extracted from the input image 
and assigns the input image to the semantic category that corresponds to a highest value of the 
similarity measure. 

16. (Original) A system as in claim 14, where said data processor computes features from the 
CFS for images in an image database; and generates a distance measure for characterizing a 
relationship of a selected image to another image in the image database by applying the 
perceptually-based similarity metric. 

17. (Original) A system as in claim 16, where values of the similarity metric computed for 
images in the image database are subsequently used to search for similar images in the image 
database. 

18. (Original) A system as in claim 16, where values of the similarity metric computed for 
images in the image database are subsequently used to organize images in the image database. 

19. (Original) A system as in claim 16, where values of the similarity metric computed for 
images in the image database are subsequently used to display images from the image 
database in an organized manner. 

20. (Original) A system as in claim 16, where said data processor cooperates with said 
graphical user interface for enabling a user to define a subset of features for the selected 
image or for an image retrieved from the image database, and subsequently uses the subset of 
features to refine a search through the image database. 
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21. (Original) A system as in claim 16, wherein the image database is located at a remote 
location and is reachable through a data communications network that is bidirectionally 
coupled to said data processor through a network interface. 

22. (Original) A system as in claim 21, where the data processor applies the perceptually- 
based similarity metric to retrieve an image from the remote image database. 

23. (Original) A system as in claim 21, where the data processor appUes the perceptually- 
based similarity metric to in conjimction with a text-based search algorithm to retrieve a 
multi-media object from the remote location. 

24. (Original) A system as in claim 14, wherein for said data processor to assign a particular 
semantic category to an image all of a set of Required Features must be present in the image, 
and at least one of a set of Frequently Occurring Features must be present in the image. 

25. (Original) A computer program embodied on a computer readable media for directing a 
computer to execute a method for processing digitally represented images, comprising 
program instructions for processing a set of perceptual semantic categories for representing 
semantic cues related to the manner in which human observers perceive and organize images, 
the semantic categories being modeled using multidimensional scaling and hierarchical 
clustering techniques and comprising a combination of perceptual features that define the 
semantics of a particular category and that discriminate that category from other categories, 
where the perceptual features and their combinations are derived through subjective 
experiments performed with himian observers; for each semantic category, program 
instructions for forming a set of the perceptual features as a complete feature set CFS and, 
responsive to an input image, program instructions for determining a CFS of the input image 



7 



S.N. 10/033,597 
Art Unit: 2621 

and for using the determined CFS to compare the input image to images stored in an image 
database. 

26. (Original) A computer program as in claim 25, where as a result of comparing the input 
image to images stored in the image database one or more most similar images are identified 
in the image database. 

27. (Original) A computer program as in claim 25, where as a result of comparing the input 
image to images stored in the image database one or more most similar images fi-om the 
image database are displayed. 

28. (Original) A computer program as in claim 25, wherein the step of using the determined 
CFS includes using a similarity metric to assign a semantic category to the input image, 
where the similarity metric operates such that all of a subset of Required Features of the 
semantic category must be present in the input image, and at least one of a subset of 
Frequently Occurring features of the semantic category must be present in the input image. 

29. -38. (Canceled) 
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